Image projection system able to emphasize or cancel objects drawn on a surface

ABSTRACT

An image projection system includes a projector to project a projected image onto a drawing surface of a whiteboard, a drawn-image detecting portion to detect a drawn image which is drawn on the drawing surface of the whiteboard while the projector is projecting the projected image, and a modification portion which, in the case where the drawn image is detected, to specify from the projected image which is projected onto the projection surface a part including at least a drawn image part overlapping the detected drawn image, and modify the specified part of the projected image, on the basis of the detected drawn image, so as to emphasize or cancel the drawn image.

This application is based on Japanese Patent Application No. 2009-213219filed with Japan Patent Office on Sep. 15, 2009, the entire content ofwhich is hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image projection system, an imageprojection method, and an image projection program embodied on acomputer readable medium. More particularly, the present inventionrelates to an image projection system which projects an image onto aprojection surface, and an image projection method and an imageprojection program embodied on a computer readable medium which areexecuted by the image projection system.

2. Description of the Related Art

In a conference or the like, an image of an explanatory material may beprojected onto a projection surface when giving an explanation.Recently, it is often the case that data of an explanatory material isstored in a computer in advance, and a projector, for example, servingas a display device is connected to the computer so as to cause theprojector to display an image of the material output from the computer.

Japanese Patent Laid-Open No. 2006-235158 discloses a projection systemwherein an image of a projection surface onto which an image has beenprojected from a first projector is picked up, a compensation parameterfor cancelling a color pattern on the projection surface is generated,and a compensation image generated on the basis of the compensationparameter is projected onto the projection surface from a secondprojector. This projection system is able to project an image as closein color to one projected onto a white screen as possible even when aprojection surface has a color pattern.

On the other hand, when a whiteboard on which a character or a graphiccan be drawn is used as a projection surface for a projector, an imagemay be drawn superposed on the image that has been projected. In thiscase, however, if a background of the image projected onto thewhiteboard is similar in lightness to the image drawn on the whiteboard,a viewer may have difficulty in recognizing the drawn image. Further, inthe case where a character or a graphic in the drawn image is superposedon a character included in the projected image, the character in theprojected image may be hard to be distinguished. Still further, in thecase where images of two or more pages are being projected, when aprojected image is switched from a first page to a second page after animage has been drawn on the first page, a newly projected image for thesecond page will be superposed on that image drawn for the first pageand left on the whiteboard, causing the projected image of the new pageto be hard to be recognized. The conventional projection systemdescribed above has not taken such problems into consideration.

SUMMARY OF THE INVENTION

The present invention has been accomplished in view of the foregoingproblems, and an object of the present invention is to provide an imageprojection system which is capable of projecting an image in such amanner that a drawn image or a projected image is readily recognizable.

Another object of the present invention is to provide an imageprojection method which enables projection of an image in such a mannerthat a drawn image or a projected image is readily recognizable.

A further object of the present invention is to provide an imageprojection program embodied on a computer readable medium which enablesprojection of an image in such a manner that a drawn image or aprojected image is readily recognizable.

In order to achieve the above-described objects, according to an aspectof the present invention, an image projection system includes: aprojection portion to project a projected image onto a projectionsurface; a drawn-image detecting portion to detect a drawn image drawnon the projection surface while the projected image is being projectedby the projection portion; and a modification portion, in the case wherethe drawn image is detected, to specify from the projected image whichis projected onto the projection surface a part including at least adrawn image part overlapping the detected drawn image, and modify thespecified part of the projected image, on the basis of the detecteddrawn image, so as to emphasize or cancel the drawn image.

According to another aspect of the present invention, an imageprojection method includes the steps of; projecting a projected imageonto a projection surface; detecting a drawn image drawn on theprojection surface while the projected image is being projected in theprojecting step; and in the case where the drawn image is detected,specifying from the projected image which is projected onto theprojection surface a part including at least a drawn image partoverlapping the detected drawn image, and modifying the specified partof the projected image, on the basis of the detected drawn image, so asto emphasize or cancel the drawn image.

According to a further aspect of the present invention, an imageprojection program embodied on a computer readable medium causes acomputer to perform the steps of; projecting a projected image onto aprojection surface; detecting a drawn image drawn on the projectionsurface while the projected image is being projected in the projectingstep; and in the case where the drawn image is detected, specifying fromthe projected image which is projected onto the projection surface apart including at least a drawn image part overlapping the detecteddrawn image, and modifying the specified part of the projected image, onthe basis of the detected drawn image, so as to emphasize or cancel thedrawn image.

The foregoing and other objects, features, aspects and advantages of thepresent invention will become more apparent from the following detaileddescription of the present invention when taken in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example of an image projection system according to anembodiment of the present invention.

FIG. 2 is a block diagram showing an example of the hardwareconfiguration of an MFP.

FIG. 3 is a functional block diagram schematically showing the functionsof a CPU included in the MFP.

FIG. 4 is a first diagram showing an example of a projected image.

FIG. 5 is a first diagram showing an example of a picked-up image.

FIG. 6A is a first diagram showing an example of a drawn image.

FIG. 6B is a first diagram partially showing a change in lightness of adrawn image.

FIG. 7A is a diagram showing an example of a first emphasizingcompensation image.

FIG. 7B is a diagram partially showing a change in lightness of a firstemphasizing compensation image.

FIG. 8A is a first diagram showing an example of a modified projectedimage.

FIG. 8B is a first diagram partially showing a change in lightnessbetween projected images before and after modification.

FIG. 9A is a diagram showing an example of a picked-up image that isobtained in the state where a modified projected image is beingprojected.

FIG. 9B is a first diagram partially showing a change in lightness of apicked-up image that is obtained in the state where a modified projectedimage is being projected.

FIG. 10 is a second diagram showing an example of a projected image.

FIG. 11 is a second diagram showing an example of a picked-up image.

FIG. 12A is a second diagram showing an example of a drawn image.

FIG. 12B is a second diagram partially showing a change in lightness ofa drawn image.

FIG. 13A is a diagram showing an example of a first cancellingcompensation image.

FIG. 13B is a diagram partially showing a change in lightness of a firstcancelling compensation image.

FIG. 14A is a diagram showing an example of a projected image beforemodification.

FIG. 14B is a second diagram showing an example of a modified projectedimage.

FIG. 14C is a second diagram partially showing a change in lightnessbetween projected images before and after modification.

FIG. 15 is a second diagram partially showing a change in lightness of apicked-up image that is obtained in the state where a modified projectedimage is being projected.

FIG. 16 is a third diagram showing an example of a projected image.

FIG. 17 is a third diagram showing an example of a picked-up image.

FIG. 18 is a third diagram showing an example of a drawn image.

FIG. 19 is a diagram showing a part of a picked-up image that isobtained in the state where a modified projected image is beingprojected.

FIG. 20 is a flowchart illustrating an example of the flow of aprojection process.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present invention will now be described withreference to the drawings. In the following description, like referencecharacters denote like parts, which have like names and functions, andtherefore, detailed description thereof will not be repeated.

FIG. 1 shows an example of an image projection system according to anembodiment of the present invention. Referring to FIG. 1, an imageprojection system 1 includes a multi function peripheral (MFP) 100serving as an image processing apparatus, a projector 200, a camera 300,and a whiteboard 400.

In image projection system 1 according to the present embodiment, apresenter at a conference stores image data which is electronic data ofdocuments for presentation, in MFP 100. Here, it is assumed that theimage data includes page data for a plurality of pages. From the pagedata for a plurality of pages included in the image data, MFP 100transmits to projector 200 page data for a page to be displayed, so asto cause projector 200 to display an image corresponding to the pagedata. The page data transmitted from MFP 100 to projector 200 may be inany format, as long as it can be read by projector 200. MFP 100transmits pieces of page data for the respective pages included in theimage data sequentially to projector 200, which in turn displays imagescorresponding to the respective pieces of page data transmitted from MFP100.

It is noted that the image data stored in MFP 100 may be data generatedby a personal computer, or data which MFP 100 obtains by reading anoriginal.

Projector 200 includes a liquid crystal display, a lens, and a lightsource, and projects an image for the page data received from MFP 100onto whiteboard 400. The liquid crystal display displays an image. Lightemitted from the light source passes through the liquid crystal displayto be projected onto whiteboard 400 via the lens. When the light emittedfrom projector 200 hits a drawing surface 401 of whiteboard 400, animage 403 which is an enlarged version of the image being displayed onthe liquid crystal display is projected onto drawing surface 401. In theimage projection system according to the present embodiment, drawingsurface 401 of whiteboard 400 also serves as a projection surface ofprojector 200.

Camera 300 is placed such that its angle of view matches drawing surface401 of whiteboard 400. Camera 300 is controlled by MFP 100 to pick up animage of drawing surface 401 of whiteboard 400 and transmit thepicked-up image to MFP 100. In the state where projector 200 isprojecting an image of page data, the picked-up image that camera 300outputs by picking up an image of drawing surface 401 includes image 403corresponding to the page data projected onto drawing surface 401, whilein the state where projector 200 is not projecting any image of pagedata, the picked-up image output from camera 300 includes a picked-upimage of drawing surface 401. Furthermore, in the state where acharacter or a graphic is drawn on drawing surface 401, the picked-upimage that camera 300 outputs by picking up an image of drawing surface401 includes a drawn image which is drawn on drawing surface 401, whilein the state where no character or graphic is drawn on drawing surface401, the picked-up image output from camera 300 includes no image drawnon drawing surface 401. Still further, in the state where projector 200is projecting an image of page data and a character or a graphic isdrawn on drawing surface 401, the picked-up image that camera 300outputs by picking up an image of drawing surface 401 includes both ofimage 403 corresponding to the page data projected onto drawing surface401 and a drawn image which is drawn on drawing surface 401.

FIG. 2 is a block diagram showing an example of the hardwareconfiguration of the MFP. Referring to FIG. 2, MFP 100 includes: a maincircuit 110; an original reading portion 130 which reads an original; anautomatic document feeder 120 which delivers an original to originalreading portion 130; an image forming portion 140 which forms, on asheet of paper or the like, a still image output from original readingportion 130 that read an original; a paper feeding portion 150 whichsupplies sheets of paper to image forming portion 140; and an operationpanel 160 serving as a user interface. Main circuit 110 includes acentral processing unit (CPU) 111, a communication interface (I/F)portion 112, a read only memory (ROM) 113, a random access memory (RAM)114, an electrically erasable and programmable ROM (EEPROM) 115, a harddisk drive (HDD) 116 as a mass storage, a facsimile portion 117, anetwork I/F 118, and a card interface (I/F) 119 which may be mountedwith a flash memory 119A. CPU 111 is connected with automatic documentfeeder 120, original reading portion 130, image forming portion 140,paper feeding portion 150, and operation panel 160, and is responsiblefor overall control of MFP 100.

ROM 113 stores a program to be executed by CPU 111 as well as datanecessary for execution of the program. RAM 114 is used as a work areawhen CPU 111 executes a program. Further, RAM 114 temporarily storesstill images that are continuously transmitted from original readingportion 130.

Operation panel 160 is provided on an upper surface of MFP 100, andincludes a display portion 160A and an operation portion 160B. Displayportion 160A is a display such as a liquid crystal display (LCD) or anorganic electro-luminescence display (ELD), and displays an instructionmenu for the user, information about acquired image data, and others.Operation portion 160B is provided with a plurality of keys, and acceptsinput of data such as instructions, characters, and numericalcharacters, according to the key operations of the user. Operationportion 160B further includes a touch panel provided on display portion160A.

Communication I/F portion 112 is an interface for connecting MFP 100 tocamera 300 and projector 200. Here, it is assumed that communication I/Fportion 112 performs serial communication with camera 300 and projector200. CPU 111 communicates with projector 200 and camera 300 viacommunication I/F portion 112, for transmission/reception of data.

Facsimile portion 117 is connected to public switched telephone networks(PSTN), and transmits facsimile data to or receives facsimile data fromthe PSTN. Facsimile portion 117 stores the received facsimile data inHDD 116, or outputs it to image forming portion 140. Image formingportion 140 prints the facsimile data received from facsimile portion117 on a sheet of paper. Further, facsimile portion 117 converts thedata stored in HDD 116 to facsimile data, and transmits it to afacsimile machine connected to the PSTN.

Network I/F 118 is an interface for connecting MFP 100 to a local areanetwork (LAN). CPU 111 can communicate, via network I/F 118, with acomputer which is connected to the LAN or to the Internet connected viathe LAN. A computer connected to the Internet includes an e-mail serverwhich transmits and receives e-mail. The LAN may be connected in a wiredor wireless manner. Furthermore, the network to which network I/F 118 isconnected is not necessarily the LAN; it may be the Internet, a widearea network (WAN), public switched telephone networks (PSTN), or thelike.

Card I/F 119 is mounted with flash memory 119A. CPU 111 is capable ofaccessing flash memory 119A via card I/F 119. CPU 111 loads a programrecorded on flash memory 119A mounted to card I/F 119, to RAM 114 forexecution. It is noted that the program executed by CPU 111 is notrestricted to the program recorded on flash memory 119A. CPU 111 mayload a program stored in HDD 116 to RAM 114 for execution. In this case,another computer connected to the LAN or the like, which is able tocommunicate with MFP 100 via network I/F 118, may rewrite the programstored in HDD 116 of MFP 100 or may additionally write a new program.Further, MFP 100 may download a program from another computer connectedto the network, and store the program in HDD 116.

The recording medium for storing the program is not restricted to flashmemory 119A. It may be an optical disk (compact disc-read only memory(CD-ROM), magnetic optical disc (MO), mini disc (MD), digital versatiledisc (DVD)), an IC card, an optical card, or a semiconductor memory suchas a mask ROM, an EPROM, an EEPROM, or the like.

As used herein, the “program” includes, not only the program directlyexecutable by CPU 111, but also a source program, a compressed program,an encrypted program, and others.

In image projection system 1 according to the present embodiment, MFP100 is connected to projector 200 and camera 300 via communication I/Fportion 112 for serial communication. Alternatively, MFP 100 may beconnected via network I/F 118, or it may perform a parallelcommunication.

FIG. 3 is a functional block diagram schematically showing the functionsof a CPU included in the MFP. The functions shown in FIG. 3 areimplemented as CPU 111 included in MFP 100 executes an image projectionprogram stored in ROM 113 or flash memory 119A. Referring to FIG. 3, thefunctions implemented by CPU 111 includes: an operation acceptingportion 51 which accepts an operation by a user; a projection controlportion 53 for controlling projector 200; an image pickup controlportion 59 for controlling camera 300; a drawn-image detecting portion61 for detecting a drawn image from within an image picked up by camera300; a modification portion 57 for modifying a projected image which isprojected by projector 200; and a character area extracting portion 55for extracting a character from a projected image which is projected byprojector 200.

Operation accepting portion 51 accepts an operation which is input intooperation portion 160B by a user, and outputs the accepted operation toprojection control portion 53 and modification portion 57. In the casewhere MFP 100 is remotely operated, operation accepting portion 51accepts an operation that a user inputs into a remote operation terminalwhich is connected to communication I/F portion 112 or network I/F 118.For example, projector 200 may be configured to function as a remoteoperation terminal. Specifically, an operation screen for a remoteoperation may be transmitted to projector 200 via communication I/Fportion 112 so that projector 200 displays the operation screen on adisplay such as an LCD. In this case, when a user inputs an operationinto projector 200 in accordance with the operation screen, projector200 transmits the input operation to MFP 100. Furthermore, in the casewhere projector 200 is connected to MFP 100 via a LAN, MFP 100 may beconfigured to serve as a Web server. In this case, when a browsingprogram is installed into projector 200, the operation screen for aremote operation can be transmitted as a Web page from MFP 100 toprojector 200. This allows projector 200 to function as a remoteoperation terminal without the need of installing therein a dedicatedprogram for remotely operating MFP 100.

Projection control portion 53 controls projector 200 connected viacommunication I/F portion 112. Projection control portion 53 transmitsan image to projector 200 to cause it to project the image. Projectioncontrol portion 53, in accordance with an operation received fromoperation accepting portion 51, reads out image data stored in HDD 116in advance. Then, projection control portion 53 selects one of aplurality of pieces of page data for a plurality of pages included inthe image data in accordance with an operation received from operationaccepting portion 51, and transmits an image corresponding to theselected piece of page data to projector 200 via communication I/Fportion 112. Herein, an image that projection control portion 53transmits to projector 200 so as to be projected thereby is referred toas a “projected image”. Operations input from operation acceptingportion 51 to projection control portion 53 herein include an operationfor specifying image data 91, a page down operation for instructing toselect page data for a next page, and a page up operation forinstructing to select page data for a previous page. Projection controlportion 53 firstly selects page data for a first page in the image dataand causes projector 200 to project a projected image corresponding tothe page data of the first page. Thereafter, projection control portion53, in accordance with an operation received from operation acceptingportion 51, causes projector 200 to project a projected imagecorresponding to the page data of a selected page.

The projected images that projection control portion 53 causes projector200 to project include, not only the images corresponding to the pagedata, but also images which are received from modification portion 57.The images received from modification portion 57 will be describedlater.

Image pickup control portion 59 controls camera 300 connected viacommunication I/F portion 112. Image pickup control portion 59 causescamera 300 to pick up an image of a projection surface of projector 200(i.e. drawing surface 401 of whiteboard 400), and receives a picked-upimage output from camera 300. Image pickup control portion 59 outputsthe picked-up image to character area extracting portion 55 anddrawn-image detecting portion 61.

Drawn-image detecting portion 61 extracts a drawn image from thepicked-up image received from image pickup control portion 59.Specifically, a difference between the picked-up image and the imagecorresponding to the page data which is a basis of the projected imagethat projection control portion 53 causes projector 200 to project, isextracted as the drawn image. Further, a picked-up image that isreceived from image pickup control portion 59 while projection controlportion 53 is causing projector 200 to project no projected image may beregarded as a drawn image. Drawn-image detecting portion 61 outputs theextracted drawn image to modification portion 57.

Character area extracting portion 55 extracts a character area includinga character, from an image corresponding to the page data which is abasis of the projected image that projection control portion 53 iscausing projector 200 to project. Character area extracting portion 55outputs the extracted character area to modification portion 57.Character area extracting portion 55 extracts a character itself as acharacter area. For extracting a character area from an image, anytechnique well known in the art may be used. For example, an image maybe divided into a plurality of blocks, and any block containing at leasta predetermined number of pixels in which a difference in lightness froman adjacent pixel is equal to or greater than a predetermined value maybe extracted. Then, in the extracted block, a collection of pixels ofwhich pixel value is equal to or greater than a predetermined value maybe extracted as a character. It is noted that a rectangular areasurrounding a character may be extracted as a character area.

Modification portion 57, on receipt of a drawn image from drawn-imagedetecting portion 61, modifies a projected image which is projected byprojector 200, and outputs the modified projected image to projectioncontrol portion 53. Modification portion 57 includes a cancellingcompensation portion 71 and an emphasizing compensation portion 73. Whendrawn-image detecting portion 61 detects a drawn image while a projectedimage is being projected, modification portion 57 activates emphasizingcompensation portion 73. In the case where projection control portion 53projects a first projected image and then projects a second projectedimage that is different from the first projected image, when drawn-imagedetecting portion 61 detects a drawn image before the second projectedimage is projected, modification portion 57 activates cancellingcompensation portion 71.

Emphasizing compensation portion 73 specifies a drawn image part wherethe drawn image received from drawn-image detecting portion 61 overlapsthe projected image, and modifies a target part in the projected image,which is made up of the drawn image part and its periphery, in such amanner that the drawn image is emphasized. Specifically, emphasizingcompensation portion 73 changes the lightness of the target part in theprojected image to lightness higher than that of a part surrounding thetarget part in the projected image. More specifically, emphasizingcompensation portion 73 generates a first emphasizing compensation imagein which the lightness of the target part is set to a predeterminedvalue and the lightness of the part other than the target part is set tozero, and combines the first emphasizing compensation image with theprojected image to thereby modify the projected image.

Emphasizing compensation will now be described. FIG. 4 is a firstdiagram showing an example of a projected image. In FIG. 4, the hatchedarea shows an area of a same color, for example blue. FIG. 5 is a firstdiagram showing an example of a picked-up image. The picked-up imageshown in FIG. 5 is obtained by camera 300 when it picks up an image ofdrawing surface 401 of whiteboard 400 after characters

and graphics including an arrow have been drawn on drawing surface 401in black, for example, in the state where the projected image shown inFIG. 4 is being projected thereon. When the image is drawn in black onthe blue area in the projected image, the handwritten charactersincluded in the drawn image may be hard to be recognized.

FIG. 6A is a first diagram showing an example of a drawn image. Thisdrawn image is extracted from a difference between the projected imageshown in FIG. 4 and the picked-up image shown in FIG. 5. FIG. 6B is afirst diagram partially showing a change in lightness of a drawn image.Referring to FIG. 6B, the drawn image part is lower in lightness thanthe part other than the drawn image part.

FIG. 7A is a diagram showing an example of a first emphasizingcompensation image. FIG. 7B is a diagram partially showing a change inlightness of a first emphasizing compensation image. Referring to FIGS.7A and 7B, in the first emphasizing compensation image, the lightness ofthe target part, made up of the drawn image part and its periphery, isset to a predetermined value, and the lightness of the part other thanthe target part is set to zero.

FIG. 8A is a first diagram showing an example of a modified projectedimage. FIG. 8B is a first diagram partially showing a change inlightness between projected images before and after modification. InFIG. 8B, a dotted line shows the lightness of the target part beforemodification, and a solid line shows the lightness of the target partafter modification. The lightness of the part other than the target partremains the same before and after the modification. Referring to FIGS.8A and 8B, the lightness of the target part is higher in the modifiedprojected image than in the projected image before modification.

FIG. 9A shows an example of a picked-up image that is obtained in thestate where a modified projected image is being projected. FIG. 9B is afirst diagram partially showing a change in lightness of a picked-upimage that is obtained in the state where a modified projected image isbeing projected. Referring to FIGS. 9A and 9B, when the modifiedprojected image shown in FIG. 8A is projected onto drawing surface 401of whiteboard 400, the drawn image that has been drawn on drawingsurface 401 comes to fall within the target part of the projected image.This increases the lightness of the area around the drawn image,allowing a viewer to readily recognize the drawn image.

Returning to FIG. 3, when operation accepting portion 51 accepts a pagedown operation or a page up operation, cancelling compensation portion71 sets an image corresponding to the page data that is an object ofprojection as a projected image, specifies a drawn image part in whichthe projected image and the drawn image received from drawn-imagedetecting portion 61 overlap each other, and modifies the drawn imagepart in the projected image in such a manner that the drawn image iscancelled. Specifically, cancelling compensation portion 71 changes thelightness of the drawn image part in the projected image to lightnesshigher than that of the area surrounding the drawn image part. Morespecifically, cancelling compensation portion 71 generates a firstcancelling compensation image in which the lightness of the drawn imageis inverted, and combines the first cancelling compensation image withthe projected image. Assuming that lightness of an image ranges fromzero to 255, in the first cancelling compensation image, the lightnessof a part other than the drawn image part is set to zero, and thelightness of the drawn image part is set to a value that is obtained bysubtracting from 255 the lightness of the drawn image part in the drawnimage.

In this manner, when an image corresponding to the page data for a newpage is projected as a projected image from projector 200, a modifiedprojected image is projected in which the part falling on the drawnimage drawn on drawing surface 401 of whiteboard 400 is high inlightness. This can make the image drawn on whiteboard 400 obscure.

Cancelling compensation will now be described. FIG. 10 is a seconddiagram showing an example of a projected image. FIG. 11 is a seconddiagram showing an example of a picked-up image. The picked-up imageshown in FIG. 11 is obtained by camera 300 when it picks up an image ofdrawing surface 401 of whiteboard 400 after characters

and graphics including an arrow have been drawn on drawing surface 401in black, for example, in the state where the projected image shown inFIG. 10 is being projected thereon.

FIG. 12A is a second diagram showing an example of a drawn image. Thedrawn image is extracted from a difference between the projected imageshown in FIG. 10 and the picked-up image shown in FIG. 11. FIG. 12B is asecond diagram partially showing a change in lightness of a drawn image.Referring to FIG. 12B, the lightness of the drawn image part is lowerthan the lightness of a part other than the drawn image part.

FIG. 13A is a diagram showing an example of a first cancellingcompensation image. FIG. 13B is a diagram partially showing a change inlightness of a first cancelling compensation image. Referring to FIGS.13A and 13B, in the first cancelling compensation image, the lightnessof a part other than the drawn image part is set to zero, and thelightness of the drawn image part is set to a value that is obtained bysubtracting from 255 the lightness of the drawn image part in the drawnimage.

FIG. 14A is a diagram showing an example of a projected image beforemodification. FIG. 14B is a second diagram showing an example of amodified projected image. FIG. 14C is a second diagram partially showinga change in lightness between projected images before and aftermodification. Referring to FIG. 14C, a dotted line shows the lightnessof the drawn image part before modification, and a solid line shows thelightness of the drawn image part after modification. The lightness ofthe part other than the drawn image part remains the same before andafter the modification. Referring to FIGS. 14A to 14C, the lightness ofthe drawn image part is higher in the modified projected image than inthe projected image before modification.

FIG. 15 is a second diagram partially showing a change in lightness of apicked-up image that is obtained in the state where a modified projectedimage is being projected. Referring to FIG. 15, when the modifiedprojected image is projected onto drawing surface 401 of whiteboard 400,the quantity of light irradiated onto the drawn image drawn on drawingsurface 401 is increased, making the drawn image obscure. The resultantimage becomes similar to the one that will be obtained when theprojected image before modification is projected onto drawing surface401 of whiteboard 400 on which no image is drawn.

Returning to FIG. 3, in the case where a drawn image is detected bydrawn-image detecting portion 61 while a projected image is beingprojected, and a character area is also received from character areaextracting portion 55, then modification portion 57 activates cancellingcompensation portion 71 and emphasizing compensation portion 73simultaneously. Specifically, when a character area is received fromcharacter area extracting portion 55, cancelling compensation portion 71specifies an overlapping part in which the drawn image part and thecharacter area overlap each other, and modifies the overlapping part insuch a manner that the drawn image is cancelled. Specifically,cancelling compensation portion 71 changes the lightness of theoverlapping part in the projected image to lightness higher than that ofthe area surrounding the overlapping part in the projected image. Morespecifically, cancelling compensation portion 71 generates a secondcancelling compensation image in which the lightness of the overlappingpart is set to a value that is obtained by subtracting from 255 thelightness of the overlapping part in the drawn image, and the lightnessof a part other than the overlapping part is set to zero, and combinesthe second cancelling compensation image with the projected image.

When a character area is received from character area extracting portion55, emphasizing compensation portion 73 specifies, from within thetarget part that is made up of the drawn image part in which theprojected image and the drawn image overlap each other and itsperiphery, a non-overlapping part that does not overlap the characterarea, and modifies the non-overlapping part in the projected image insuch a manner that the drawn image is emphasized. Specifically,emphasizing compensation portion 73 changes the lightness of thenon-overlapping part in the projected image to lightness higher thanthat of a part surrounding the non-overlapping part in the projectedimage. More specifically, emphasizing compensation portion 73 generatesa second emphasizing compensation image in which the lightness of thenon-overlapping part is set to a predetermined value and the lightnessof a part other than the non-overlapping part is set to zero, andcombines the second emphasizing compensation image with the projectedimage that has been combined with the second cancelling compensationimage by cancelling compensation portion 71, to thereby modify theprojected image.

A cancelling and emphasizing compensation process will now be describedin detail. FIG. 16 is a third diagram showing an example of a projectedimage. In FIG. 16, the hatched area shows an area of a same color, forexample blue. FIG. 17 shows an example of a picked-up image. Thepicked-up image shown in FIG. 17 is obtained by camera 300 when it picksup an image of drawing surface 401 of whiteboard 400 after characters

and a circular graph are drawn on drawing surface 401 in black, forexample, in the state where the projected image shown in FIG. 16 isbeing projected thereon.

FIG. 18 is a third diagram showing an example of a drawn image. Thedrawn image is extracted from a difference between the projected imageshown in FIG. 16 and the picked-up image shown in FIG. 17. FIG. 19 is adiagram showing a part of a picked-up image that is obtained in thestate where a modified projected image is being projected. In FIG. 19,the overlapping part where the drawn image part and the character areaoverlap each other is shown by fine hatching, and the non-overlappingpart, which is a part of the target part made up of the drawn image partand its periphery and which does not overlap the character area, isshown by coarse hatching. The lightness of the overlapping part shown bythe fine hatching is higher than the lightness of its surrounding area.This causes the drawn image to be cancelled, so that the character canbe readily recognized. Furthermore, the lightness of the non-overlappingpart shown by the coarse hatching is higher than the lightness of a partother than the non-overlapping part in the target part, whereby thedrawn image is readily recognizable.

FIG. 20 is a flowchart illustrating an example of the flow of aprojection process. The projection process is carried out by CPU 111included in MFP 100 as CPU 111 executes an image projection programstored in ROM 113 or flash memory 119A. Referring to FIG. 20, CPU 111reads image data 91 stored in HDD 116 (step S01). CPU 111 specifies andreads image data 91 stored in HDD 116, in accordance with an operationinput into operation portion 160B by a user, or in accordance with anoperation input into a remote operation terminal by a user in the casewhere MFP 100 is remotely operated from the remote operation terminal.

Next, CPU 111 sets an image corresponding to page data for a first pageincluded in the image data as a projected image (step S02). Then, CPU111 causes projector 200 to project the projected image (step S03). Atthis time, the image corresponding to the page data for the first pageis projected onto drawing surface 401 of whiteboard 400.

In the following step S04, CPU 111 controls camera 300 to pick up animage of a projection surface (i.e. drawing surface 401 of whiteboard400). Then, it acquires a picked-up image output from camera 300 (stepS05). CPU 111 then determines whether a drawn image has been detected(step S06). A difference between the projected image projected in stepS03 and the picked-up image acquired in step S05 is regarded as a drawnimage. If a drawn image has been detected, the process proceeds to stepS07; otherwise, the process proceeds to step S18.

In step S07, CPU 111 extracts a character from the projected imageprojected in step S03. Then, CPU 111 determines whether the drawn imageextracted in step S06 and the character extracted in step S07 overlapeach other (step S08). Specifically, it is determined whether the drawnimage part including the drawn image and a character area including thecharacter overlap each other. If the drawn image part and the characterarea overlap each other, the process proceeds to step S09, so as tocarry out a cancelling and emphasizing compensation process. If thedrawn image part and the character area do not overlap each other, theprocess proceeds to step S14, so as to carry out an emphasizingcompensation process. When no character is extracted from the projectedimage, it is determined that the drawn image and the character do notoverlap each other, and the process proceeds to step S14.

In step S09, an overlapping part is specified. The overlapping part is apart in which the drawn image part and the character area overlap eachother. Then, a non-overlapping part is specified (step S10). Thenon-overlapping part is a part of the target part, made up of the drawnimage part and its periphery, that does not overlap the character area.In the following step S11, a second cancelling compensation image forcancelling the drawn image is generated. The second cancellingcompensation image is an image in which the lightness of the overlappingpart is set to a value that is obtained by subtracting from 255 thelightness of the overlapping part in the drawn image, and the lightnessof the part other than the overlapping part is set to zero.

Then, in step S12, a second emphasizing compensation image foremphasizing the drawn image is generated. The second emphasizingcompensation image is an image in which the lightness of thenon-overlapping part is set to a predetermined value, and the lightnessof the part other than the non-overlapping part is set to zero. In thefollowing step S13, the second cancelling compensation image generatedin step S11 and the second emphasizing compensation image generated instep S12 are combined with the projected image projected in step S04 tothereby generate a combined image. Then, CPU 111 causes projector 200 toproject the combined image (step S17). At this time, while the drawnimage is drawn on drawing surface 401 of whiteboard 400, a modifiedprojected image is projected in which the overlapping part where thedrawn image part and the character area overlap each other is higher inlightness than its surrounding area and, in the target part made up ofthe drawn image part and its periphery, the non-overlapping part thatdoes not overlap the character area is higher in lightness than itssurrounding area. As a result, in the case where a drawn image and acharacter overlap each other, a projected image can be projected so asto make both the drawn image and the character distinguishable.

On the other hand, in step S14, a target part is specified. The targetpart is a part made up of the drawn image part and its periphery. In thefollowing step S15, a first emphasizing compensation image foremphasizing a drawn image is generated. The first emphasizingcompensation image is an image in which the lightness of the target partis set to a predetermined value and the lightness of the part other thanthe target part is set to zero. In the following step S16, the firstemphasizing compensation image generated in step S15 is combined withthe projected image projected in step S04 to generate a combined image,and the process proceeds to step S17. In step S17, CPU 111 causesprojector 200 to project the combined image generated in step S16. Atthis time, while the drawn image is drawn on drawing surface 401 ofwhiteboard 400, a modified projected image is projected in which thetarget part made up of the drawn image part and its periphery is higherin lightness than its surrounding area. As a result, a projected imagecan be projected so as to make the drawn image readily distinguishable.

In step S18, it is determined whether a page down operation has beenaccepted. If so, the process proceeds to step S19; otherwise, theprocess returns to step S04. While the page down operation is describedhere, the operation which is accepted is not limited thereto; it may bea page up operation or any other operation for switching a projectedimage.

In step S19, CPU 111 causes projector 200 to stop projection of animage. Then, CPU 111 controls camera 300 to pick up an image of theprojection surface (i.e. drawing surface 401 of whiteboard 400) (stepS20), and acquires a picked-up image output from camera 300 (step S21).Then, among the plurality of pieces of page data included in the imagedata read in step S01, an image corresponding to the page data for thepage immediately following the page that has been processed till then isset as a projected image (step S22).

In step S23, it is determined whether a drawn image has been detected.Specifically, it is determined whether the picked-up image acquired instep S21 includes a drawn image. If a drawn image has been detected, theprocess proceeds to step S24; otherwise, the process returns to stepS03.

In step S24, a first cancelling compensation image for cancelling adrawn image is generated. The first cancelling compensation image is animage in which the lightness of the drawn image part including the drawnimage is set to a value that is obtained by subtracting from 255 thelightness of the drawn image part in the picked-up image, and thelightness of the part other than the drawn image part is set to zero.

Then, in step S25, the first cancelling compensation image generated instep S24 is combined with the projected image set in step S22 togenerate a combined image. The combined image is set as a new projectedimage (step S26), and the process returns to step S03. In step S03, CPU111 causes projector 200 to project the new projected image, or, inother words, the combined image obtained by combining the firstcancelling compensation image with the image corresponding to the newpage data. At this time, while the drawn image is drawn on drawingsurface 401 of whiteboard 400, a modified projected image is projectedin which the drawn image part is higher in lightness than itssurrounding area. This causes the drawn image to be cancelled, wherebythe projected image for the new page becomes readily recognizable.

As described above, according to image projection system 1 of thepresent embodiment, when an image drawn on drawing surface 401 ofwhiteboard 400 is detected while a projected image is being projectedthereon, the projected image that is projected onto the projectionsurface is modified on the basis of the drawn image. Accordingly, it ispossible to project a projected image which is modified so as toemphasize or cancel the drawn image drawn on drawing surface 401 ofwhiteboard 400.

Specifically, a target part, which is made up of a drawn image partwhere the projected image projected onto drawing surface 401 ofwhiteboard 400 and the drawn image overlap each other and a periphery ofthe drawn image part, is modified so as to emphasize the drawn image.This allows the drawn image drawn on drawing surface 401 of whiteboard400 to be readily recognized.

More specifically, the target part in the projected image is modified soas to have lightness that is higher than the lightness of an areasurrounding the target part. This increases the lightness of the areaaround the drawn image, allowing the drawn image to be separated ordistinguished from the projected image, whereby the drawn image becomesreadily recognizable.

In the case where a projected image includes a character area in which acharacter is included, an overlapping part where the character areaoverlaps the drawn image part in which the projected image and the drawnimage overlap each other is modified so as to cancel the drawn image. Inaddition, a non-overlapping part, which is included in the target partmade up of the drawn image part and its periphery and which does notoverlap the character area, is modified so as to emphasize the drawnimage. As a result, the projected image can be modified in such a mannerthat both the character included in the projected image and the drawnimage are readily recognizable.

Specifically, the overlapping part is modified so as to have lightnessthat is higher than the lightness of an area surrounding the overlappingpart. As a result, the lightness of the overlapping part is increased,causing the quantity of light irradiated onto the drawn image to beincreased. This makes the drawn image obscure and, hence, the characterreadily recognizable.

Further, the non-overlapping part is modified so as to have lightnessthat is higher than the lightness of an area surrounding thenon-overlapping part. As a result, the lightness of the area around thedrawn image is increased, enabling the drawn image to be readilyrecognized.

When a projected image is switched to another image, in the other image,a drawn image part that would overlap the drawn image is modified so asto cancel the drawn image. This causes the drawn image to be cancelled,making the other image readily recognizable.

Specifically, the lightness of the drawn image part is changed to alevel higher than the lightness of the area surrounding the drawn imagepart, so as to cancel the drawn image. This increases the quantity oflight irradiated onto the drawn image, thereby making the drawn imageobscure.

While image projection system 1 in the above-described embodiment iscomposed of MFP 100, projector 200, and camera 300, MFP 100 may bereplaced with a personal computer as long as it can control projector200 and camera 300. Furthermore, projector 200 may be configured to havecamera 300 built therein and have the functions shown in FIG. 3, inwhich case image projection system 1 may be implemented by projector 200alone.

Furthermore, the present invention may of course be understood as animage projection method for performing the process shown in FIG. 20 oras an image projection program for causing a computer to perform theimage projection method.

Although the present invention has been described and illustrated indetail, it is clearly understood that the same is by way of illustrationand example only and is not to be taken by way of limitation, the spiritand scope of the present invention being limited only by the terms ofthe appended claims.

What is claimed is:
 1. An image projection system comprising: aprojection portion to project a projected image onto a projectionsurface; a drawn-image detecting portion to detect a drawn image drawnon said projection surface while said projected image is being projectedby said projection portion; and a modification portion, in the casewhere said drawn image is detected, to specify from said projected imagewhich is projected onto said projection surface a part including atleast a drawn image part overlapping said detected drawn image, andmodify the specified part of said projected image, on the basis of thedetected drawn image, so as to emphasize said drawn image, wherein saidmodification portion includes an emphasizing compensation portion whichspecifies a target part including the drawn image part in which saidprojected image being projected onto said projection surface and saiddetected drawn image overlap each other and a periphery of the drawnimage part, and modifies said target part so as to emphasize said drawnimage.
 2. The image projection system according to claim 1, wherein saidemphasizing compensation portion modifies said target part so as to havelightness higher than lightness of an area surrounding said target part.3. The image projection system according to claim 1, further comprisinga character area extracting portion to extract a character areaincluding a character from said projected image which is projected,wherein said modification portion includes a cancelling compensationportion, in the case where said character area is extracted, to specifyan overlapping part where said extracted character area overlaps thedrawn image part in which said projected image being projected onto saidprojection surface and said detected drawn image overlap each other, andmodify said overlapping part so as to cancel said drawn image, and anemphasizing compensation portion, in the case where said character areais extracted, to specify, from a target part including the drawn imagepart in which said projected image being projected onto said projectionsurface and said detected drawn image overlap each other and a peripheryof the drawn image part, a non-overlapping part that does not overlapsaid extracted character area, and modify said non-overlapping part soas to emphasize said drawn image.
 4. The image projection systemaccording to claim 3, wherein said cancelling compensation portionmodifies said overlapping part so as to have lightness higher thanlightness of an area surrounding said overlapping part.
 5. The imageprojection system according to claim 3, wherein said emphasizingcompensation portion modifies said non-overlapping part so as to havelightness higher than lightness of an area surrounding saidnon-overlapping part.
 6. The image projection system according to claim1, wherein said modification portion includes a cancelling compensationportion which specifies a drawn image part in which said projected imageand said detected drawn image overlap each other, and modifies saiddrawn image part so as to cancel said drawn image.
 7. The imageprojection system according to claim 6, wherein in the state where saidprojection portion projects a first projected image and a secondprojected image successively in this order, in the case where saiddrawn-image detecting portion detects a drawn image before said secondprojected image is projected, said cancelling compensation portionmodifies said second projected image.
 8. The image projection systemaccording to claim 6, wherein said cancelling compensation portionmodifies said drawn image part so as to have lightness higher thanlightness of an area surrounding said drawn image part.
 9. An imageprojection method comprising the steps of: projecting a projected imageonto a projection surface; detecting a drawn image drawn on saidprojection surface while the projected image is being projected in saidprojecting step; and in the case where said drawn image is detected,specifying from said projected image which is projected onto theprojection surface a part including at least a drawn image partoverlapping said detected drawn image, and modifying the specified partof said projected image, on the basis of the detected drawn image, so asto emphasize said drawn image, wherein said modifying step includes anemphasizing compensation step of specifying a target part including thedrawn image part in which said projected image being projected onto saidprojection surface and said detected drawn image overlap each other anda periphery of the drawn image part, and modifying said target part soas to emphasize said drawn image.
 10. The image projection methodaccording to claim 9, wherein said emphasizing compensation stepincludes a step of modifying said target part so as to have lightnesshigher than lightness of an area surrounding said target part.
 11. Theimage projection method according to claim 9, further comprising thestep of extracting a character area including a character from saidprojected image which is projected, wherein said modifying step includesa cancelling compensation step of, in the case where said character areais extracted, specifying an overlapping part where said extractedcharacter area overlaps the drawn image part in which said projectedimage being projected onto said projection surface and said detecteddrawn image overlap each other, and modifying said overlapping part soas to cancel said drawn image, and an emphasizing compensation step of,in the case where said character area is extracted, specifying, from atarget part including the drawn image part in which said projected imagebeing projected onto said projection surface and said detected drawnimage overlap each other and a periphery of the drawn image part, anon-overlapping part that does not overlap said extracted characterarea, and modifying said non-overlapping part so as to emphasize saiddrawn image.
 12. The image projection method according to claim 11,wherein said cancelling compensation step includes a step of modifyingsaid overlapping part so as to have lightness higher than lightness ofan area surrounding said overlapping part.
 13. The image projectionmethod according to claim 11, wherein said emphasizing compensation stepincludes a step of modifying said non-overlapping part so as to havelightness higher than lightness of an area surrounding saidnon-overlapping part.
 14. The image projection method according to claim9, wherein said modifying step includes a cancelling compensation stepof specifying a drawn image part in which said projected image and saiddetected drawn image overlap each other and modifying said drawn imagepart so as to cancel said drawn image.
 15. The image projection methodaccording to claim 14, wherein said cancelling compensation stepincludes a step of, in the state where a first projected image and asecond projected image are projected successively in this order in saidprojecting step, in the case where a drawn image is detected in saiddetecting step before said second projected image is projected,modifying said second projected image.
 16. The image projection methodaccording to claim 14, wherein said cancelling compensation stepincludes a step of modifying said drawn image part so as to havelightness higher than lightness of an area surrounding said drawn imagepart.
 17. An image projection program embodied on a non-transitorycomputer readable medium, the program causing a computer to perform thesteps of: projecting a projected image onto a projection surface;detecting a drawn image drawn on said projection surface while theprojected image is being projected in said projecting step; and in thecase where said drawn image is detected, specifying from said projectedimage which is projected onto the projection surface a part including atleast a drawn image part overlapping said detected drawn image, andmodifying the specified part of said projected image, on the basis ofthe detected drawn image, so as to emphasize said drawn image, whereinsaid specifying includes an emphasizing compensation which specifies atarget part including the drawn image part in which said projected imagebeing projected onto said projection surface and said detected drawnimage overlap each other and a periphery of the drawn image part, andmodifies said target part so as to emphasize said drawn image.